Learning Trees and Rules with Set-Valued Features

In most learning systems examples are represented as xed-length \feature vectors", the components of which are either real numbers or nominal values. We propose an extension of the feature-vector representation that allows the value of a feature to be a set of strings; for instance, to represent a small white and black dog with the nominal features size and species and the set-valued feature color, one might use a feature vector with size=small, species=canis-familiaris and color=fwhite,blackg. Since we make no assumptions about the number of possible set elements, this extension of the traditional feature-vector representation is closely connected to Blum's \innnite attribute" representation. We argue that many decision tree and rule learning algorithms can be easily extended to set-valued features. We also show by example that many real-world learning problems can be eeciently and naturally represented with set-valued features; in particular , text categorization problems and problems that arise in propositionalizing rst-order representations lend themselves to set-valued features.